About the Program

Data Pilot is specially created to expand data analysis options available in Microsoft Excel with nonparametric statistical methods.

The statistical methods used in the application offers the most relevant results even with small numbers of samples (3 or more observations), without any assumptions on distribution of the original sample. This is especially important when analyzing biological information or when doing expensive or difficult to replicate experiments.

The tight integration with Microsoft Excel allows users take advantage of the full potential of the program for entering data and creating reports. Data Pilot works with Excel 97, 2000, XP and, most likely, the future versions of Excel.


Basic Data

The sample size can not be less then 3 observations. There is no upper limit. The demo version takes the first 12 samples.

The data can be located in either rows or columns (radiobutton “Grouping”).

If there are no sample names, the program gives them the default names of Var1, Var2 … VarN.

The table with data has to be separated from the rest by empty row and column or the sheet edges.

Example

To input basic data and indicate the location for the results, you have to:

  1. Select any non-empty data cell in the Microsoft Excel sheet;
  2. At then program's “Data” tab:

    1. Set “Names” checkbox;

    2. Select the “Grouping” radiobutton value;

    3. Press the “Input” button.

After you do this, Excel with select the data area. Its coordinates will be recorded below the “Input” button.

Note. Please remember that the data inputs right when the button “Enter” is pressed. If the data has changed, you have to enter it again.

To indicate where the results should be entered, select any cell in the Microsoft Excel sheet and press “Output” button.

Right after this, the coordinates of the top left cell of the results area will appear below the “Output” button. Take a note that the calculation method selection field and the “Next >>” button that executes it became available.

Note. Please keep in mind that the contents of the cells to the right and below will be overwritten with the calculation results.

The results can be located on the same sheet, a different one or in a new Excel book.


Examples of data preparation and processing

All original basic data for all examples are located in the file Example.xls, located in the program folder.

Take a note of the fact that your results (besides the Taxon Value) may vary from the third or fourth decimal. This is due to probablistic nature of the calculation methods.

All sample data is taken from the book “Southern Russia Grape Sorts” (Krasnodar: “Volnye mastera”, 2001 – 192.c). The authors are L.P. Troshina, P.P. Radchevskiy, A.I. Mislivsky (www.vitis.ru).


Statistics methods

Statistical methods

Historically, most statisticsal methods have been developed before the computer era. Back then it was very difficult to do large sample calculations. That is why the methods that used hypothesis about sampled population distribution became widespread. However, it is quite difficult to correctly apply distribution-dependent methods. And using these methods without indicating appropriate distribution models leads to the erroneous results.

Monte-Carlo method appeared then, but did not get popular due to a large amount of calculations required. Now, when the computers are so available, two methods based are on it – bootstrap method and shuffling method. They allow getting reliable results regardless of the original (basic) data. Jackknife can also provide accurate results, but require a large sample size.

Distribution-dependent methods drawbacks

If we know that a sampled population has a normal distribution, then a small sample is enough to estimate its average and dispersion, which makes it easy to get the confidence interval, compare it to the averages from other samples and so on. However, there are the following problems here:

  1. A large sample size needs for a reliable check of distribution. Using Kolmogorov-Smirnov’s criteria it requires 100 or more observations. In reality, such a large sample size is a rarity. Fifteen observations are enough to apply Chi-Square test, but it is less accurate and highly depends on group splitting. Moreover, frequently there are less then 15 observations.
  2. Even when there are many observations, they are most often not distributed normally. In that case, it takes a lot of time and diligent effort to normalize data through various data conversion methods. Unfortunately, sometimes normalization is impossible, for example for the multimodal distributions.

Bootstrap method

The advantage of this method is that instead of one statistical parameter value it allows to get sample distribution for this parameter.

For example, when we have 10 pairs of observations, using the classic method, we get only 1 correlation coefficient. The bootstrap method uses random subsamples to determine a new correlation coefficient each time. When we get many values for this parameter, we build a distribution curve for it in order to estimate its significance. When about half of the values are positive and half are negative, the correlation is insignificant. If, however, 95% of the correlation coefficients are positive, we can state that there is positive correlation dependence between the variables with the confidence probability of 0.95. Frequently, instead of confidence probability, people use significance level, which is calculated by subtracting the confidence probability value from 1; in our case, it’s 0.05.

Shuffling method

This method is used in order to determine the probability of correlation between variables. The original data is used to calculate correlation coefficient. Then the value of one of the variables gets “shuffled” and the statistical values get calculated again and again. This way we get several statistical values. If we get the values of correlation coefficients that are just as high as with the original data – then there is no correlation between the variables. If, however, the resulting values are lower, then the correlation is assumed and a confidence level can be determined.

Jackknife method

This method is similar to the bootstrap method except a subsample selection. In the above example, if we had 10 observations we would get 10 selections with 9 observations in each (if we throw out 1 observation) or 10*9=90 subsamples with 8 observations in each, if the throw out 2 observations.


Confidence interval

The bootstrap method can be used to calculate confidence interval for the average from each sample for the given confidence level. Confidence probability is calculated by subtracting the significance level value from one. The subsamples may vary in sizes.

Use this method to compare an average with a constant. For comparing one average with an average from another subsample, use comparing averages technique.

Subsamples with 3-4 observation can be used only for the preliminary data analysis. In order to achieve confidence probability 0.9 or above you have to have no less then five samples; for 0.95 – at least 6 samples, for 0.99 – 8 or more.

Note. If the lower line for the subsample has a confidence probability lower that the set value, then, due to small sample size, it is impossible to determine the confidence interval for the set confidence probability.

Example

After indicating the basic data area and the results area, select from the dropdown menu “Confidence Interval” and press the "Next >>" button.

Here you have to select confidence probability value (one minus the significance level) and the number of repetitions. More repetitions mean higher precision level for the results and longer calculations time.

The "<< Back" button allows users to come back to the original data and calculation method selection.

Press the “Calc” button to start calculations.

During the process of calculation, a progress window appears. The “Cancel” button allows user to stop the calculations. After the calculation is complete, you get the results.

You can compare the results by using Compare Samples option. In spite of the fact that 90% confidence intervals for “C Mag” and “M 49” intersect, there is a 98% probability that the “M49” average is higher than the “C Mag” average.

This method allows comparing averages with constants; for example, it can be used to determine with grape sorts have average harvest value more than 15. Take a note of the fact that with a confidence probability of 0.9 we can not say that of the sort “M control”.


Comparing Averages

The bootstrap method allows comparing averages of the subsamples in order to determine the probability of one average is higher than the other. The sizes of the subsamples may differ.

Sometimes, confidence intervals are used for comparing averages, but this is the wrong approach. There are instances when 90% confidence intervals coincide, yet one average is larger than the other with very high probability. One of such cases is described below.

Subsamples from 3-4 observation can be used only for the preliminary data analysis. In order to approach the confidence probability of 0.99 you have to have at least 8 data points in two subsamples.

Example

After indicating the basic data area and the results area, select from the dropdown menu “Confidence Interval” and press the "Next >>" button.

Select here the number of iterations. More iterations mean higher precision level for the results and longer calculations time.

The "<< Back" button allows users to come back to the original data and calculation method selection.

Press the “Calc” button to start calculations.

During the process of calculation, a progress window appears. The “Cancel” button allows user to stop the calculations. After the calculation is complete, you get the results.

You can compare the results you got with the Confidence Interval. Despite the fact that 90% confidence intervals for “C Mag” and “M 49” coincide, the average for “M 49” is higher than the average for “C Mag” with the probability of 98%


Value of correlation

The significance of a correlation

The shuffling method allows us to determine the significance of a correlation between the variables measured with different scales. The attributes can be measured by the naming scale (start with “n:”), order scale (“o:”), or interval scale (start with “i:” or do not have “:”). Different attribute values are defined by the whole numbers.

As a result, you get a confidence probability value, which is calculated by subtracting the value of significance level from one.

The samples must have the same size.

If you compare the significance of a correlation of attributes with a variable, measured with the naming scale, the results can be interpreted as the attributes values for the discriminant analysis (classification with a learning sample), where the variable defines splitting of the learning sample into groups.

The results vary from 0 (no correlation) to 1 (proven correlation) and are interpreted as a confidence probability (one minus the significance level value).

If necessary, the results can be presented as a correlation matrix between the matrixes of attributes similarity.

Example

After indicating the basic data area and the results area, select from the dropdown menu “Confidence Interval” and press the "Next >>" button.

You need to enter here the number of iterations and indicate if the correlation matrix needs to be displayed. More iterations mean higher precision level for the results and longer calculations time.

The "<< Back" button allows users to come back to the original data and calculation method selection.

Press the “Calc” button to start calculations.

During the process of calculation, a progress window appears. The “Cancel” button allows user to stop the calculations. After the calculation is complete, you get the results.


Importance for classification

The method allows comparing attributes based on their importance for classification without a learning sample – performing cluster analysis.

The attributes can be measured by the naming scale (start with “n:”), order scale (“o:”), or interval scale (start with “i:” or do not have “:”). Different attribute values are defined by the whole numbers.

The samples may have different sizes, but in order to compare the significance of attributes between each other, small samples must have the same size. Larger samples (over 10 observations) may vary in their sizes.

The values for the importance for classification may vary from 0 (no correlation) to 1 (proven correlation). The results are on the order scale, meaning we can state that one attribute is more valuable than the other, but can not say by how much.

Example

After indicating the basic data area and the results area, select from the dropout menu “Confidence Interval” and press the "Next >>" button.

This method does not require setting any parameters.

The "<< Back" button allows users to come back to the original data and calculation method selection.

Press the “Calc” button to start calculations.

During the process of calculation, a progress window appears. The “Cancel” button allows user to stop the calculations. After the calculation is complete, you get the results.


System Requirements:

  • PC-compatible computer with processor Pentium-100 or better;

  • Microsoft Windows 95/98/NT/2000/ME/XP;

  • Microsoft Excel 97/2000/XP.


How To Register?

You can buy Data Pilot at a cost of $ 55.55.

You can use one of the following ways:

  • Internet (Online);

  • Phone;

  • Fax;

  • Postal mail.

To order the program, please go to the page: http://www.colorpilot.com/datapilot.html. If you do not receive the registration information within 1-2 business days after paying, or if you have any problems with ordering, please do not hesitate to contact us at support @colorpilot.com.


Why To Register?

When you register the program, you will be able to:

  • Contact our tech support;

  • Process samples with more than 12 observations.


Technical Support

If you have a problem with Data Pilot and need help, please contact technical support staff at: datapilot@colorpilot.com. Please inform us about the following:

  • Data Pilot version (from About dialog);

  • Where did you get Data Pilot (http or ftp site);

  • Windows version, US or international, OEM or not;

  • Description of your problem (as much information as possible to retrieve the problem).

We'll try to help you as fast as possible, usually in one or two business days.

If you have any comments or suggestions for the next releases, please feel free to post them to us.


Our Developments

For office:

Form Pilot Home (Pro)™ - for filling out paper forms of any type (instead of a typewriter).

Form Pilot Office™ - for filling out electronic forms of any type (pdf, doc, xls, txt...)

PDF Filler Pilot - fills in PDF forms.

PDF Maker Pilot - for making PDF documents and fillable PDF forms.

Type Pilot™ - type frequently used text quick and easy!

Voice E-Mail Pilot™ - say what you think and send it by e-mail!

Document converters:

Document2PDF Pilot™ - Converts MS Word, Excel, HTML, RTF and many other documents to PDF files.

CHM2PDF Pilot™ - Converts CHM documents to PDF files.

HTML2PDF Pilot™ - Converts HTML documents to PDF files.

TEXT2PDF Pilot™ - Converts RTF and text documents to PDF files.

CHM2HTML Pilot™ - Converts CHM into a set of HTML files.

Image converters:

Image2PDF Pilot™ - Batch conversion utility that converts images (TIFF, JPG, PNG, EMF) to PDF document.

TIFF2PDF Pilot™ - Batch conversion utility that converts images TIFF files (including multi-page images) to PDF documents.

PDF2TIFF Pilot™ - Converts PDF files to Tiff.

Universal converters:

PDF Printer Pilot (Pro)™ - Convert any document to PDF document using this virtual printer.

PDF2Mail Pilot (Pro)™ - Convert files to PDF and send them by email.

Virtual Print Pilot™ - Convert any documents to pictures.

For digital camera users:

Color Pilot™ - for quick and easy color correction - for Windows and Macintosh.

Color Pilot Plug-in™ - plugin (for Photoshop, PhotoDeluxe, Photo-Paint, Paint Shop Pro...) for quick and easy color correction.

Color Pilot Junior™ - this invention makes color correction simple for children.

Print Pilot™ - for quick and easy printing graphic images on the printer.

Silver Pilot™ - converts negative photos into positives.

Secret Photos™ - turn your photos into puzzles and create gifts for your friends!

Slide Show Pilot™ - creates slide shows and screensavers.

For Internet:

Notes Pilot™ - handy notes organizer for Internet.

Educational software:

tPilot™ - creates different tests, processes and evaluates results.

tPilot Home™ - allows you to create tests and quizzes for your friends and family.

Games:

Flash Games Collection (for Windows and Macintosh) - collection of eight cute flash games.

Useful utilities:

iPilot™ - musical player, placed in the system tray, supports skins and controls WinAMP.

Sound Pilot™ - add typewriter sounds to your computer keyboard. You can use different sound schemes!

Tray Pilot™ - allows you to hide the System Tray icons.

Text2Log Print Pilot™ - Specially designed to store documents using plain text.

For software developers:

Flash Editor - flash projectors editing software.

PDF Creator Pilot - is the library for creating PDF files from Visual Basic, ASP, Delphi, Visual C++, VBScript.

Text Pilot Library - draw text in your applications at any angle and any direction.

Notes Pilot Lite™ - handy notes organizer for Internet, lite version, freeware!

Surf Safe Pilot™ - utility which can help you browse Internet with absolute security.

Traces Viewer™ - shows traces that were made by different web-sites in your Internet Explorer, freeware!

Educational software:

tPilot™ - creates different tests, processes and evaluates results.

tPilot Home™ - allows you to create tests and quizzes for your friends and family.

Useful utilities:

iPilot™ - musical player, placed in the system tray, supports skins and controls WinAMP.

MagicMap™ - specifically designed for preparing raster image like maps for archiving, vectorization or direct plotting (color recognition), freeware!

Sound Pilot™ - creates the romantic sound atmosphere as it was at the times of the typewriters. At every touch of the keyboard there is the new sound, which makes the process of typing more interesting, amuses and reduces stress.

Tray Pilot™ - allows you to hide the System Tray icons.

Tray Pilot Lite™ - allows you to hide the System Tray icons, lite version, freeware!

Virtual Print Pilot™ - converts any documents to pictures.

Products for software developers:

Flash Tuner - software for modification of flash-projectors.

Pilot PDF Creator - an ActiveX programming library for creating PDF files.

Text Pilot Library - draw text in your applications at any angle and any direction.